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ABSTRACT 



Issues in the design of research to describe 
educational practice are examined, and arguments for and against 
particular methodologies are summarized. A single study is used for 
illustrative purposes; it explores the relationship between English 
taught as a second language in achool and English language needs in 
the workplace. The study uses both ethnographic and quantitative 
research techniques. The report first outlines the variables in the 
study. A discussion of procedures describes the need analysis, 
sampling, establishment of a systematic classroom observation 
schedule, audio recording of classroom interaction, interviews with 
teachers and students, and examination of teacher records. Rationales 
for these and additional procedures are then explained. The 
subsequent section describes construction of the classroom 
observation instrument and the method for merging quantitative 
observational data with data drawn from the audio recording. The 
conducting of interviews is then discussed, including interview 
types, interviewer attributes, questions, problems, and 
administrative issues. A 45-item bibliography is included. (MSE) 
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Constructing a Multi-Dimensional Research Study 



Introduction 

This paper is about preparing for studies that aim both to describe educational 
practice and to understand causes and effects (Ball, 1991:188). It summarises 
arguments for and against particular research instruments for such purposes. The 
particular study used to Illustrate this explores language 1n the workplace 
related to the language curriculum 1n schools 1n Malaysia, but the general 
research Issues discussed may be relevant to others engaged 1n explanatory, In- 
depth Investigations. Consequently the paper describes a number of approaches 
to research, each aiming to provide a different angle of approach to the specific 
problem being Investigated, and constitutes a more detailed account of some 
research methods, to follow on from the first Centre for Language 1n Education 
Occasional Paper (Brumflt and Mitchell, 1990). 

The Malaysian research explores the relation between school practice and expected 
workplace needs, using both quantitative and ethnographic techniques. It is not 
pure ethnography, because the amount of time available to spend 1n both school 
and workplace settings is insufficient to justify such a description, but similar 
techniques of observation and analysis are used, in the sense described by Lutz 
(1993: 108): 

Ethnography is a holistic, thick description of the interactive 
processes Involving the discovery of Important and recurring 
variables 1n the society as they relate to one another, under 
specified conditions, and as they affect or produce certain results 
and outcomes in the society. It is not a case study, which narrowly 
focuses on a single Issue, or a field survey that seeks previously 
specified data, or a brief encounter (for a few hours each day for 
a year, or 12 hours a day for a few months) with some group. Those 
types of research are ethnographic but not ethnography! 

Hammersley (1993) reports that when the limitations of positivism for the social 
sciences were realized, many educational researchers turned to ethnography. 
However, as ethnography involves long periods of immersion 1n the field, which 
most educationists who hoped to obtain quick answers to educational problems 
cannot afford to do, case studies utilizing ethnographic techniques became 
Increasingly popular by making the following modifications: 

1. the amount of time 1n a setting 1s reduced, on the grounds that it is 
necessary to produce results rapidly to be useable, so that the Information 
does not get out of date; 

2. access 1s negotiated on the basis that those studied control the data 
collected, and must give permission for Its publication. 

In looking at the curriculum, a "matching" study (Eggleston et. al. 1975: 55) may 
be appropriate. The use of a standardized observation schedule with different 
classes will make possible comparisons between the classes Involved, using 
quantitative methods of recording and analysis. In addition, a questionnaire 
survey can also be used, providing further quantitative evidence. 

To justify use of these quantitative techniques, 1t 1s necessary to Identify the 
variables of the study so as to provide some guidelines for the construction of 
instruments. 

With this general Introduction 1n mind, let us consider specific procedures in 
relation to the specific research questions. 
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Variables in research 



The main research question for this project was whether school practice matched 
perceived English language needs for the workplace 1n school. The main concepts 
to be operational zed were the concept of needs for the workplace 1n addition to 
the concept of school practice . These two form the overarching concepts of the 
study. 

With regard to the first, the concept of needs 1s made up of target needs, 
process needs, and Individual subjective needs. 

In the analysis of needs for the workplace, the variables are' broken down Into 
types of jobs, language related work tasks, settings for language use, levels of 
target performance, dialect use, and registers. The variables of topics and 
content of Interaction, language exponents, role relationships within the work 
context, workplace culture and other contextual variables will be obtained 
through Interviews rather than questionnaire survey as certain Information will 
be specific to the Individual and cannot be included 1n a comprehensive way for 
every member of staff, as they are difficult to predict. 

As part of the systematic observation of the classroom, the concept of school 
practice can be broken down into the curriculum, the teaching, the learning 
experience, the classroom context and supporting resources and students' current 
level of achievement in the subject. 

The identification of variables will help to structure the quantitative aspects 
of the study, and aid in the construction of indicators to be observed and 
quantified. 



THE RESEARCH PROCEDURES 



The needs analysis component 

The needs analysis component will utilize the procedures outlined below to obtain 
a definitive account of the types and range of occupations that need English in 
carrying them out. At the same time it will provide information on the functional 
use of English in the workplace and the language skills that are needed in the 
execution of these occupations. 

1. Analysis of job advertisements in the English language dallies of 
Malaysia, the levels of proficiency expected, and the specification of 
spoken, or written English, and other skills specifications, e.g. 
presentation skills, etc. 

2. Questionnaire survey of a small number of firms 1n the 

business and industrial sector towards the Identification of needs by 
employers 

3. Observation of three contrasting occupations in the workplace, 
Including depth- interviewing of the respondents 1n two different 
private sector companies 

4. Depth interviews of three other workers in contrasting 
occupations in the same two companies. 

The school sample will comprise two classes each in two schools, one a mixed race 
school in an urban area, and the other a predominantly ethnic Malay school in a 



semi urban area. Form Four classes win be chosen, one good class and one average 
one 1n each school. Since the observation will occur towards the end of the 
school year, 1t would not be advisable to select Form Five classes as they will 
be busy with revision work for the school certificate examination. A period of 
two weeks each will be spent in each school for the systematic recording and 
audio-recording, with an additional two days each for ethnographic-! 1ke 
observation. 

So the following procedures will be utilized: 

5. A systematic classroom observation schedule , which will record the 
occurrence of categories pertinent to the study such as workplace-like 
classroom behaviour and activities and topical content of workplace 
Interactions. This is felt to be a very suitable procedure to adopt when 
aiming at the analysis and study of the match between school experience and 
perceived workplace needs, since the variables related to workplace needs 
would have been established prior to the school visits. 

6. Audio-recording , which will be used 1n qualitative retrospective 
analysis, with the aim of drawing out relevant language behaviours related 
to perceived workplace competencies and in the hope of being able to 
establish theoretical constructs from evidence contained in the classroom 
transcripts. 

It could also be used as a checking device to supplement and reinforce the 
researcher's notes, records and tallies. 

Additionally, in order to determine process needs and teachers' teaching 
philosophy, methodology and attitudes as well as to clarify events observed in 
the teaching-learning processes, and explore teacher and students' interactions, 
the following will be utilized. 

7. In-depth unstructured interviews with teachers. 

In order to get at students' subjective needs, their attitudes and feelings 
towards English, as well as their self-assessment of their ability in English, 
the following will be utilized: 

8. In-depth unstructured interviews with students. 

In order to establish students' proficiency in the English language: 

9. An examination of teacher's assessment records throughout the year. 

In order to obtain other contextual information, teacher's record books, schema 
of work for the year, support materials and equipment for English available in 
the school will also be examined. 

Additionally, 1t is hoped that contextual qualitative data will reveal certain 
patterns and relationships to explain any observed relationships between language 
learning processes in the classroom and the level of English proficiency of the 
students. 



As the main thrust 1s towards a case study approach, using ethnographic 
techniques, where 1t 1s hoped theory will evolve as the data unfolds, no 
hypotheses will be developed or tested 1n the research procedures. 



RATIONALE FOR THE PROCEDURES USED 



Rationale for the analysis of job advertisements 

In undertaking the analysis of job advertisements, 1t 1s hoped to obtain a 
significant account of people looking for jobs who are conversant 1n English and 
therefore read English newspapers, and would be able to perform jobs 1n which 
English 1s used. In addition to giving Information on the range of occupations 
that require English to a greater or lesser degree, 1t 1s envisaged that at the 
same time the target level of proficiency required for the various occupations 
as well as the specific skills that are stressed for the three contrasting 
occupations studied 1n depth, will be clarified to a greater degree, and thus 
provides insights Into the world of work that the teacher in the classroom hoping 
to prepare her/his students to operate 1n, has no Insight Into, These 
advertisements provide a rich source of information on workplace needs, and 
represent a source that has not been utilized 1n the construction of curriculum 
for schools. 



Rationale for the questionnaire survey of workplace needs 

The questionnaire survey 1s undertaken to clarify hunches regarding the use of 
English in the workplace. The researcher would already have certain 
preconceptions regarding the use of English In the workplace (such as those 
gleaned from the literature and from the analysis of advertisements), and the 
survey 1s undertaken to confirm or disconfirm these hunches. 

A mailed questionnaire 1s the easiest, most economical and fastest way to cover 
a broad sample for purely descriptive information. To glean the nature of 
workplace English use, one general questionnaire 1s administered to a small 
sample of firms, and covers all categories of workers 1n those firms analyzed on 
the basis of location, within the Klang Valley and outside of the Klang Valley. 
Ideally, different occupations or different categories or levels of occupations 
should merit one questionnaire, but questions of practicality rule out this 
posslblity. Different businesses should also receive different questionnaires, 
but, 1t was not possible for the researcher to predict the type of occupations 
or the nature of tasks related to specific businesses. As such, the decision was 
taken to use one questionnaire for all types of firms, covering all levels of 
workers. 

However, to overcome the obvious limitations of the mailed (general) 
questionnaire, additional, qualitative data will be collected from workplace 
observation of the three workers 1n different occupations, and 1n-depth 
Interviewing of those observed together with three other workers, which would 
also allow Inter and Intra company comparisons. 



Rationale for using a case study approach with ethnographic techniques 



As reported 1n Hammersley (1993), pos1t1v1st1c techniques 1n research drawn from 
psychology with Its experimental and correlational paradigms, were Influential 
1n classroom observation studies (p. x). This pos1t1v1st1c tradition was 
challenged by the writings of Blumer, among others, who argued that rather than 
human actions being merely responses to psychological conditioning or social 
forces, they were in fact "actively constructed on the basis of complex processes 
of Interpretation through which people make sense of the world. These processes 
derive from, but also act back upon, perspectives which are developed 1n the 
course of social Interaction" (p, x1). Both Blumer, from the symbolic 
interaction 1st school, and ethnomethodologlsts who also argued that "human 
actions cannot be understood without taking Into account of the processes of 
interpretation which generates them" (p. x11) advocate the use of ethnographic 
techniques to study patterns of social interaction, including those taking place 
in classrooms. I agree with this view, as the classroom unit is a dynamic and 
fully functioning social unit of its own with Its own culture, and any study of 
it 1s to be understood in the way Its members see and understand their everyday 
happenings and interactions. 

Phenomenology studies "direct experience taken at face value and sees behaviour 
as determined by the phenomena of experience rather than by external, objective 
and physically described reality" (Cohen and Manion, 1985: 327). It is a reaction 
against the belief that human behaviour is determined by psychological 
conditioning or social forces (Hammersley, 1986: x1). 

Ethnography 1s concerned with the study of social interaction in natural settings 
through observation and informal participation and conversation (Hammersley and 
Atkinson, 1983). Roman and Apple (1990) asserts that ethnography renders 
"..•'thick' contextual descriptions of social subjects as they actively and 
creatively make sense of their social worlds (citing Geertz, 1973)" and was a 
response to "structuralist reproduction theories to grossly abstract and 
overtheorize about what goes on 1n the daily life of cultural institutions or 
informal settings, whether they are schools, families, or youth subcultures" (p, 
44), 

Ethnomethodology is concerned with how people make sense of their everyday world. 
It treats meanings as context-dependent in which agreement among the actors is 
not produced by commonality of meanings but rather by the collective use of 
shared methods to create intelligible behaviour and to find meaning in other's 
actions (Cohen and Manion, 1985: 32 and 33). It treats common everyday life as 
a phenomenon 1n Its own right, 

Interactionlsm focuses on the nature of Interaction: "Individuals Interact; 
societies are made up of Interacting individuals. People are constantly 
undergoing change 1n Interaction and society 1s changing through interaction" 
(Cohen and Manion, 1985: 35). 

To quote Hammersley, what 1s Important 1s that both Interactionlsm and 
ethnomethodology encourages the detailed qualitative analysis of patterns of 
social Interaction including those taking place 1n classrooms (Hammersley, 1986: 
x11). 

Furthermore, various interdisciplinary Influences from psychology, sociology and 
linguistics have caused a shift from the "quantitative analysis of large samples 
towards detailed, qualitative investigations of smaller amounts of data, even of 
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single lessons" (p. x11). This has given rise to a proliferation of case studies 
which utilize ethnographic techniques as well as quantitative procedures. And in 
view of the time constraints of the proposed study this seems the most sensible 
choice of procedure to adopt. 



Rationale for the observation of three workers in their workplace and the 
observation of four classes 

In the proposed study, observation in the natural setting of the workplace is to 
be undertaken, together with observation 1n the natural setting of the classroom. 
Although half of the classroom observation will utilize a systematic Instrument, 
the other half will not depend on any structured methodology, with the commitment 
to discover this methodology in the course of being 1n the setting and studying 
1t. 

The case study is done in an ethnographic mode, in an endeavour to draw out 
grounded theory in the sense used by Glazer and Strauss (1967) and explicated by 
various writers of educational research (Woods, 1985). Ethnography (and 
ethnographic techniques) lend themselves well to grounded theorizing as they 
study how people Interact in a given setting as it 1s Influenced by their 
beliefs, values, perspectives, motivations and how they interpret each other's 
actions. In ethnography, researchers endeavour to look from the inside, into how 
these groups of people have constructed their highly distinctive cultural 
realities (Woods, 1986: 4-5), rather than impose their armchair theories on the 
people they had hitherto no contact with. 

Woods (1985:52) explains how it is done. 

The main emphasis is on discovery rather than testing of theory, but 
analysis is sequential - it is both guided by and guides data 
collection. Categories and their properties are noted and 
'saturated*. Concepts emerge from the field, are checked and 
rechecked against further data, compared with other material, 
strengthened or perhaps re-formulated. Models of systems are built 
up in the process of research, and gradually a theory comes into 
being with its distinctive characteristic of explanation and 
prediction linking the revealed concepts into an integrated 
framework, the operationalization of which has been demonstrated. Its 
plausibility may be strengthened by further case studies in the same 
area, though it may be continuously refined since it must accomodate 
all data, and not simply answer to a 'majority of cases'. Later the 
level of abstraction might be raised by this 'substantive' theory 
becoming 'formal* theory, as case studies from other substantive 
areas are compared and examined for common elements. 

The reason why the research has no ready hypotheses at the outset is because 1t 
is conducted by going into the research situation as far as possible with an open 
mind, with a commitment to understanding the situation and in the hope of being 
able to formulate explanations as to why certain phenomena occur on the tasis of 
observations. Bailey (1978: 55) concurs with this view of grounded theorizing 
following a sequence of entering the field, describing what happens, tlnd 
formulating explanations based on what is seen and understood. 
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Error and bias 1n ethnographic research 



Undeniably, ethnographic and sociological research can give rise to bias and 
errors 1f the research does not keep a tight control not only during the data 
collection process, but also during the data analysis process. Phillips (1971) 
reported that 1n some sociological studies when responses are checked against 
other sources of Information such as official records, 1t was often found that 
respondents do not give truthful answers. These response errors are frequently 
found 1n studies such as those asking for health information or questions about 
deviant behaviour. Furthermore, bias can arise out of the researcher's own 
attributes such as race, sex and age. There is also respondent bias as it is 
usual for respondents to deny socially undesirable traits and to admit to 
socially desirable ones. Researchers unwittingly have a tendency to project their 
expectations onto the person being studied, and for reasons of soda! 
desirability respondents tell the researcher what they think the researcher wants 
to hear. 



Ref lexivity 

Being reflexive means doing a self analysis of the methodology chosen as the 
study unfolds. All good ethnographic accounts should attempt to be reflexive by 
recounting the research experience, the decisions made, and the rationale 
underlying those decisions. According to Hammersley (1983:4), by being reflexive 
an obligation is placed on researchers to make themselves aware of decisions they 
are taking and the motives that underlie those decisions Including the obligation 
to monitor those decisions and the willingness to make adjustments to decisions 
already taken. Hammersley claims that much of the literature on ethnography is 
more concerned with the rhetorical justification of ethnography rather than a 
systematic analysis of the research process. 

Lately, however, there have been studies that try to document the ethnographer's 
experience reflexively, for example Hitchcock (1983) in the study of an open plan 
school; Measor and Woods (1983) 1n the study of the myths that confront new 
students as they transfer from middle school to upper school, as part of their 
discovery of a new social reality and their adaptation to it; and Beynon (1983) 
on the problems of being accepted as a member of the society being studied. 

Data analysis for ethnographic study 

Undoubtedly, ref lexivity occurs at all stages of the research process, and 
researchers shows their accountability to the reader by recounting all stages of 
the process. Reflexivlty operates at the level of data analysis, by discussing 
the problems of making inference and providing evidence 1n ethnographic data 
collected through participant observation (Becker, 1978). Becker notes that there 
were problems to be resolved at each stage of data analysis - from the first 
selection and definition of problems, concepts and Indicators such as the 
accuracy of Initial conclusions and the credibility of informants; to the second 
stage of being accurate in reaching certain conclusions about the frequency and 
distribution of certain phenomena; and on to the construction of partial models 
of the social system before integrating them Into an overall model, researchers 
face decisions about the validity of their conclusions and the weight of evidence 
that can convince the reader. One way of dealing with this problem is to describe 
the "natural history" of drawing conclusions so that readers will be able to make 
their own judgement "as to the adequacy of the proof and the degree of confidence 
to be assigned the conclusion" (pp. 322-323). 
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Various writers (such as Walford, 1991 and Measor and Woods, 1991) have urged the 
recounting of the research process. Measor and Woods (1991) for example think 
that, since most research does not proceed as smoothly the reports seem to 
Indicate, it is a good Idea to report breakthroughs and blockages and "decision 
points" (p. 61), especially for the benefit of novice researchers, who might blame 
themselves if things do not proceed smoothly. 



Rationale for using systematic observation 1n the classroom 

Before we can justify using a systematic observation procedure for the classroom, 
we will need to consider the tradition of systematic observation, and the issues 
that are involved in using this method, so that any disadvantages and weaknesses 
of the method can be addressed. 



Review of systematic observation procedures 

At present, there is no instrument available in the literature that seeks to find 
the relationship between workplace needs and classroom teaching, even though many 
of these Instruments contain aspects that are relevant to the proposed study. 
This part of the paper will report on these instruments as reviewed in Croll 
(1986) and will borrow features from them that are relevant. 

The "One in Five" is aimed at investigating whether teacher-student interactions 
of children with special needs differ from those of normal children. It employs 
a time-based recording at every ten second interval and the procedure was to 
observe each child for a few minutes, each time moving from child to child. The 
ORACLE PROJECT (Observational Research and Classroom Learning Evaluation) aims 
to describe and analyze student and teacher activities and interactions in the 
primary school classroom and to relate these to the progress children were making 
in their school achievements. It focuses on the behaviour and interaction of 
students (not more than a sample of 8) and teachers. It employs a time-based 
recording every twenty-five second Intervals for the students and a period of 
observation for the teacher 1n any one lesson. Two records are kept, a teacher 
record and a student record. 

The focus of observation for "One in Five" as well as the "Oracle" are a few 
select students only, while for the classic model, FIAC (Flanders Interaction 
Categories), 1t is the teacher Interacting with the class as a whole. It is 
difficult to see how coding 1s done when the lesson is organized for group 
activity, and different groups may be doing different activities, such as one 
group actively talking and interacting aand another group doing individual silent 
reading. Here 1s where FIAC falls us; it 1s more suited for classes with a 
traditional orientation, a point made by Walker and Adelman (1993), who claim 
that it sees "teacher-student Interaction 1n terms of the transmission of 
Information ... (and) ... does not concern itself with talk as the expression and 
negotiation of meanings ..." (p. 7). Further criticisms put forward by Delamont 
and Hamilton (1976) should be mentioned also, such as the focus on discrete bits 
of behaviour making 1t difficult to go beyond the categories and thus impeding 
theoretical development (p. 8-9). 

T he Teacher's Day aims to get a representative account of the working day of 
teachers in junior schools. The observation focuses on the teacher's activities 
during the day and teacher's work commitment. A continuous recording 1s employed 
in which a record 1s kept of the broad features of teacher's work, and changes 
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1n the teacher's activity are recorded, Instead of noting down the activity the 
teacher is engaged in at certain time intervals, as FIAC does. 



Flanders* Interaction Analysis Categories (FIAC) aims to measure teaching 
effectiveness based on teacher's direct and indirect influence; indirect 
influence 1s said to have a positive relationship with learner achievement. The 
observation is focused on teacher talk and student talk and a continuous 
recording every three seconds is made. 

"A study of schooling" aims to study what happens 1n classrooms including 
physical environment. Four different observation procedures are employed which 
are focused on the physical environment, the teaching-learning activities that 
transpired during the observation, the details of student-adult interactions and 
the fourth record concerns space, materials and decision making. The observer 
spends a whole day 1n the classroom, tackling each procedure in turn, and going 
through several cycles of the first three procedures, while the last procedure 
is recorded last. 

The reason why these various systematic schedules are reviewed is because their 
design and construction give a number of indications of how a new one can be 
developed for the purposes of the proposed study. The variables and categories 
looked at, the structure of the instrument, the type of recording used are all 
helpful in designing an observation instrument that contains the features that 
has been tried and validated, and has worked and yielded significant quantifiable 
results. In emulating them novice researchers can avoid the pitfalls that can 
arise from relying solely on their own judgement when exemplars are available, 
and are discussed, their weaknesses pointed out and thus avoided, or other 
criteria included to overcome these weaknesses (for example the absence of 
context variables). 



Criticisms of systematic observation 

Hammersley (1986:xiii) reports on the main criticisms of systematic observation, 
which are: 

1. The use of pre-established categories prevents recognition of the 
complexity of classroom behaviour and obstructs the development of theories 
that are sensitive to this complexity. 

2. By using arbitrary time sampling, systematic observation neglects 
natural patterns in classroom interaction. 

3. Classroom interaction 1s studied without any attempt to understand the 
context in which it occurs and in particular the perspectives of the 
teachers and pupils involved. 

The weaknesses cited by Hammersley above are valid if systematic observation is 
used in Isolation from other technqiues. However, in the proposed study, the 
combination of ethnographic techniques and the practice of reflexivity in the 
research process it is hoped will overcome most of the weaknesses mentioned. 

The use of a systematic observation schedule is defensible where comparisons are 
to be made; as a matching exercise is Involved 1n this study, 1t 1s necessary to 
specify some predetermined criteria to do the match against. But the defence of 
the systematic instrument rests on the extent to which its categories have been 
successfully justified 1n relation to the rich ambitions of th* study. 
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CONSTRUCTING THE OBSERVATIONAL INSTRUMENT 



The categories of the systematic observation schedule 

To begin the job of constructing the schedule, indicators are developed which 
form the categories in the observation schedule. Variables are operational ized 
into sets of categories into which occurrences can be coded unambiguously 
according to the rules and criteria which define them. 

According to Croll (1986:62): 

The variables used reflect the processes by which we turn the theoretical 
concepts which underlie our attempts to describe aspects of classrooms into 
a set of working definitions which are used to provide actual descriptions. 

According to Burgess (1986:60), these sets of working definitions of variables 
provide low inference measures in the coding as they reduce the role of observer 
judgement to a minimum. Sometimes high inference variables are used in which 
judgements cannot easily be made in terms of predetermined criteria (such as in 
TALOS, Ullmann and Geva, 1984:127), leaving the observer to make a personal 
response to what is seen. High inference variables usually use rating scales. 

Burgess claims that concepts are developed by theorists and variables and 
indicators are developed by researchers. Variables in this sense are "a set of 
definitions whereby an instance to be coded can be placed uniquely into one of 
these categories." 

With regard to the study in question, this structure can be exemplified 
diagrammatically, as shown below. 



CONCEPTS 



VARIABLES 



INDICATORS 



Perceived workplace needs 

School practice 

The match between the two 



Target needs 

Process needs 

Subjective needs 

Curriculum 

Language system 

Classroom context 

Students' language proficiency 

Workplace competencies and behaviours 

Classroom activities 

Topics 

Affect (attitudes, etc.) 
Language use 
Language awareness 
Organization 

(Class, group or individual) 
Physical context: seating, 

equipment, etc* 

Cultural context: shared meanings 
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The development of a systematic observation schedule for the study 

Of course, no available and validated observation scheme will do for research 
with a specific focus, as the variables outlined above Illustrate. In designing 
a research instrument, 1t 1s Important to remember that 1t should be capable of 
getting Information on the research concerns and the questions to be answered and 
should represent a culmination of research Ideas operational 1 zed on the basis of 
a theoretical construct. Sampling decisions as well as data analysis are 
interlinked with the decisions taken about the recording procedures adopted. 

However, Croll (1986:50) makes the following point when a decision has been made 

to design a new schedule: 

when a researcher decides that the research questions- being addressed 
require a new system rather than using an existing one it will almost 
certainly be of value to incorporate some aspects of a well established 
schedule. 

CrolT's reasons were mainly because some "bench-mark* 1 measures will be available 
which can be used for comparison, but also because the research itself will be 
part of a cumulative activity, benefiting from such previous experience as is 
relevant to the new study. 

Of the schedules- reviewed, some are capable of being imitated in structure and 
perhaps some of the observational categories, but in the end 1t is the aims of 
the study that determine the content of the Instrument. That was the reason it 
was decided to develop a separate systematic observational schedule specific to 
the needs of the study, but at the same time emulating the design and some of the 
features of schedules available in the literature. 

For this study, a close reading and study of the COLT (Communicative Orientation 
of Language Teaching) which 1s aimed at distinguishing communicatively oriented 
classroom from non-communicative ones as well as a close reading and study of 
TALOS (Target Language Observation Scheme) (both in Ullmann and Geva, 1984) which 
was used in the formative evaluation of a French second language programme, were 
the main schedules utilized in the development of the systematic observation 
instrument. The structure of the schedule is borrowed from. COLT. Some of the 
categories (indicators) were also taken from COLT as well as from TALOS. 

At the same time, the schedules reported above have been utilized to a lesser 
extent, thus some of the indicators or categories used are from the One 1n Five, 
and the recording system is adapted from The Teacher's Day. 



Defining the categories 

M T1me" and "activity" relate to the choice of type of recording, which is 
e vent/act 1v1ty-based. This 1s to say that a record is made each time a new 
activity is introduced by the teacher, and subsequently the other categories are 
filled in with each change in activity and with each classroom event. Should 
there be no change of event after five minutes of the last one, the categories 
will be filled 1n after every five minutes. 

Since the thrust of the observation is on whether workplacelike behaviour, or 
competencies that have been identified as being essential for the workplace are 
present in the classroom practice, the main part of the schedule consists of 
these workplacelike behaviours. 
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The language behaviours utilize Brumflt's (1984:70) main categories, I.e. 

- Conversation/discussion 

- Comprehension 

• Extended writing 

as they are a more Integrated classification that are closer to workplace 
language use rather than the more lockstep classification of the traditional four 
skills. Apart from these, the detailed categories are entirely based on the 
researcher's judgement of what constitutes workplace competencies under Brumflt's 
broad headings and it 1s felt the operational Ization does not conflict with his 
concept of them. 

In addition to these workplace competencies, the topical content of activities 
are recorded as being related or unrelated to workplace needs, and 1f necessary, 
a note made in the "contextual information and other related explanation" column 
as to what these topics are. 

Apart from these very directly linked workplace needs, others that are related 
tn a contextual sense to language use in communication also form part of the 
coding system. These are: one, attention to the language as a system, or as what 
1s commonly referred to now as language awareness with its breakdown; actual 
language use in class, i.e. "teacher talk" and "student talk", which is hoped to 
show either a balance or the preponderance of one in relation to the other. 

The next category is classroom organization, again to show the balance of 
students' active involvement in language work in paired or group interaction, 
(i.e. fluency practice), as compared to a more passive role when students are 
involved in a whole-class activity. 

Lastly, to see whether affective development is being catered for, since the 
results of the analysis of Job advertisements have shown that certain affective 
skills, such as leadership and social and interpersonal skills, are Important in 
the workplace (and are communicated through language), a separate column, called 
"affective development" is included. This is to be distinguished from "affect" 
with regard to affective behaviour towards the target language, which will be 
obtained from students' Interviews. 



Focus of observation and type of recording 

In addition to the detailing of the above, Croll (1986) stresses that an 
observation schedule must have a clear focus of observation within an overall 
context of observation. The focus of observation is usually the teacher or 
certain identified students, or as in the case of FIAC, teachers while 
interacting with the students. The context of observation is the classroom as a 
whole. In the study in question, the focus of observation is the students as they 
undergo their learning experiences. Below are given the decisions made as to the 
essential features of the schedule. 

Unit of analysis - The classroom 

Focus of observation - Students and their learning experiences 

Type of recording Activity-based, event recording, 

i.e. any changes 1n classroom activity 1s 
recorded, including a record made of the 
time this change occurred; 1f no change in 

13 
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activity occurs five minutes after the 
last one activity, a shift 1s made to a 5 
minute time-based recording 

Context variables are noted down 1n the "contextual variables and other related 
explanation" column. The recording of context variables takes care of some of the 
limitations that have beset some of the well-known systematic Instruments, such 
as FIAC. Their recording will have to be done through notations because their 
lack of predictability make 1t difficult to define categories for them 1n the 
systematic schedule. Thus, the attention to contextual variables in this manner 
will enable a better understanding of the events that occur in relation to the 
total functioning of the class as a social unit 



The qualitative component of the classroom observation through audio-recording 

Although notes will be taken simultaneously with the recording of the systematic 
schedule, they are not likely to be adequate as the researcher's attention will 
be almost completely involved in filling in of the various categories of the 
schedule. Therefore, the bulk of the qualitative Information would be obtained 
from the tape-recorded lessons, which will be transcribed and analysed. It 1s 
upon this transcribed data that some form of ethnographic analysis will be 
followed possibly modelled on the coding procedures used in the University of 
Southampton KAL (Knowledge about Language) project (Mitchell, Hooper and Brumfit, 
1994). 

In addition, it is proposed to emulate aspects of the analysis that is used by 
Adams and Biddle (1970) 1n the project called Realities of Teaching, reported in 
Cohen and Manion (1985: 143-146)). Adams and Biddle utilized two kinds of 
measurement, one, the incident count, which registers each Instance when the 
different kinds of behaviour occurred, and two, the duration count, which 
registered the time-span of each particular Incident. From the data, Adams and 
Biddle were able to draw up lesson profiles based on student and teacher roles, 
location and function of activities, and whether participants in interpersonal 
exchanges were emitters or targets. Although the proposed study in all 
probability will not utilize lesson profiles in its secondary analysis, Realities 
of Teaching does provide a good example of how observational data can be 
structured and yield data "that are objective and capable of quantification" 
(Cohen and Manion, 1985: 146). 

This, however, would mean that there are two sets of data that will have to be 
merged in some acceptable way so as to contribute to the greater significance of 
the overall results. 



Merging data from two styles of observation, systematic and ethnographic 

In analysis, the two sets of data can be treated 1n three ways as suggested by 
Galton & Delamont (1985:172): 

1. The sets of data could be treated equally 

2. The qualitative data could be treated as more valid 
and the quantitative used to reinforce important and/or 
controversial points 1n the argument. 

3. The quantitative data could be regarded as "the facts" and the 
qualitative used to "flesh them out", illustrate them, or "humanize" 
them. 
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At this point 1n the research the feeling 1s that method number 1 would be 
utilized. However, the ultimate decision would rest on the qualities of the data 
themselves, as undoubtedly, the best way to approach them would be to capitalize 
on their respective strengths and richness. 



TEACHERS AND STUDENTS 1 INTERVIEWS 



Interviews as a research strategy 
Types of Interviews 

There are basically three types of interviews: the structured, semi-structured 
and unstructured. The structured Interview 1s very much like a questionnaire, but 
is administered by the interviewer rather than being self-administered. It is 
chosen when the responses do not require too much reflection, and the questions 
are not contentious. The semi-structured interview contains some structured 
questions followed by probes, and recording is usually done on the form itself. 
Unstructured depth interviews require considerable skill to administer and should 
produce information, by the use of probes, which might not otherwise have 
emerged. Such Interviewers are usually trained and experienced (Wragg, 1984: 183- 
185). Depth Interviews often carry the same connotation as unstructured 
interviews. 

Powney & Watts (1987:119) writing on the potential of depth interviews have this 
to say: 

For a depth interview, which allows the best possible exploration of views 
and behaviour patterns, the interview is adapted to the Individual and no 
set questions may be asked, although the interviewer usually has a set 
agenda to cover. The aim is to get informants to talk freely and openly 
about themselves, only initial stimuli being provided by the interviewer. 
In this case, 1t is more appropriate to have a 11st of possible topics. The 
discussion could range well beyond these, provided the interviewer feels 
it is relevant to the overall aims of the research. 

For the study concerned, teachers' and students' interviews are planned using the 
strategy of unstructured depth interviewing. This is the preferred Interview 
style for this study, as apart from factual and descriptive data it 1s hoped to 
be able to draw out attitudes, Interest, commitment and other affective states 
of both teachers and students. 

Burgess (1984) at the time of writing his book found a considerable amount of 
literature on structured interviews, but little on unstructured interviews, so 
the material for the chapter on interviewing 1n his book on field research was 
drawn mainly from his experience 1n using interviews as a research strategy. 

He perceives interviews as conversations, in which the interviewer acts as a 
friend and confidant and tries to build up trust in the interviewee. It is useful 
when Interviewing a group with certain characteristics to have an interviewer who 
has undergone the experiences that they have had because this would contribute 
to understanding and rapport. What is vital is to form a comfortable relationship 
with the subject being interviewed (Burgess, 1984:103-107). 
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Interviewer's attributes 



Powney and Watts (1987) have a lot of useful hints for the would-be interviewer, 
too. Like Burgess, they emphasize the importance of establishing rapport. One 
way of being sure of getting the respondent's attention, is to establish from the 
beginning the amount of time the interview will take as well as to assure the 
respondent of confidentiality. It is good practice to ask the structured 
questions first, to give the respondent time to get used to the situation and the 
interviewer's presence. The interviewer must have credibility for the interviewee 
to take the exercise seriously, and lastly Powney and Watts say that "As far as 
possible the interviewer should give the impression of talking, rather than 
quizzing" (p. 133), echoing Burgess' (1984) views about interviews as 
conversations. 

Cohen and Manion (1985: 310), reviewing Woods (1986) on ethnography, sum up the 
attributes of Interviewers in ethnographic studies, as produced below: 

Trust: There would have to be a relationship between the interviewer and 
interviewee that transcended the research, that promoted a bond of 
friendship, a feeling of togetherness and joint pursuit of a common mission 
rising above personal egos. 

Curiosity: There would have to be a desire to know, to learn people's views 
and perceptions of the facts, to hear their stories, discover their 
feelings. This is the motive force, and 1t has to be a burning one, that 
drives researchers to tackle and overcome the many difficulties involved 
in setting up and conducting successful interviews. 

Naturalness: As with observation one endeavours to be unobstrusive in order 
to witness events as they are, untainted by one's presence and actions, so 
in interviews the aim is to secure what is within the minds of the 
interviewees, uncoloured and unaffected by the Interviewer. 

Some of the Interviewer skills suggested by Powney and Watts (1987) are 
establishing logical order for questioning; good listening skills; good retention 
of what has been said, so that it can be followed up later in the interview; 
being perceptive to (non-linguistic) paral inguistic cues; having empathy towards 
the interviewee; being adaptable to people and circumstances; being aware of the 
impression one's appearance creates and being prepared to change negative 
appearance or behaviour; being comfortable with the agenda of the interview that 
has been planned; being able to handle difficult questions; being prepared to 
prompt and probe; being able to handle silence in the interview, so as to allow 
time to the interviewee to formulate his/her thoughts; accurate recording; being 
able to make a decision of whether to end or continue the interview, as for 
example, 1f a previously reticent Interviewee becomes voluble when the 
interviewer begins to leave; and lastly, making certain that the Interviewee 1s 
left 1n a calm state should the Interview had been upsetting 1n some way (e.g. 
asking parents about the education and future of their severely handicapped 
child) (Powney and Watts, 1987:134-140). 

It is Important to have an "aide memolre" or agenda for the Interview mainly to 
ensure consistency across all Interviews so that similar Information is collected 
to enable comparison across cases. The aide memolre or agenda contains a list of 
topics, themes and questions although what appears in 1t may not be the actual 
questions that would be asked. Apart from good listening skills, interviewers 
should be careful over their words, gestures and comments so as not to advance 
or impede the interview, and also not to interrupt so as to allow a smooth flow 
of talk (Burgess, 1984:108 -111). 
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Types of Interview questions 



Burgess (1984: 111-112, citing Spradley, 1979) delineates three types of 
questions. 

First, descriptive questions which allow informants to provide statements 
about their activities. Secondly, structural questions which attempt to 
find out how informants organize their knowledge and, finally, contrast 
questions which allow informants to discuss their meanings of situations 
and provide an opportunity for comparisons to take place between situations 
and events in the Informants' world. These particular questions are used 
at different points in the Interview while the interviewer probes for 
details about the Informants and encourages them to to discuss situations 
in their own terms. 

Burgess gives examples of transcripts with a commentary at the side to illustrate 
his use of the three types of questions, as well as 1 11 ust rating other interview 
strategies such as keeping the interview flowing, repeating, making a link 
between comments, etc. 



Problems with using interviews 

Burgess (1984) further discusses problems 1n unstructured interviewing such as 
"question wording, bias, rapport and avoiding loaded questions" (p. 119). Apart 
from these, there are other problems. 

Researchers need constantly to monitor the direction, depth, and detail of 
the interviews, recording, transcription and subsequent analysis" (p. 120). 

Time management is also important, not only with regard to total length of the 
interview (not more than two hours; one and a half is considered optimum), and 
making estimates about blocks of time to devote to particular topics (Burgess ; 
1984:120). 

Regarding the problem of bias, Cohen and Manion (1985) point out that the sources 
of bias are the characteristics of the interviewer, the characteristics of the 
respondent, and the substantive content of the questions. There are potential 
dangers in attitudes and opinions, and often misperceptlons and misunderstandings 
occur. Potential sources of bias are colour, religion, social class and age. 
However, it is possible to reduce and control bias by careful formulation of 
questions ensuring that the meaning is clear, also by the probability sampling 
of respondents, rigorous training of interviewers and ensuring that interviewer 
characteristics match with Interviewee characteristics. 

Another problem is Invalidity. This can also be controlled by creating rapport 
and trust so that the Interviewee is not guarded and would more readily reveal 
information that would not otherwise have been revealed in more formal 
circumstances or if the Interviewee 1s more guarded (Cohen and Manion, 1985: 
319). 

Wragg (1984) tells of the pitfalls to avoid 1n the conduct of interviews. These 
are: 

- Interview bias, i.e. framing questions in a way that leads the 

interviewee in certain directions, either consciously or subconsciously 
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- Sample bias, I.e. not being representative 



- Problem of using hired Interviewers who may fake answers when 
interviewees are unco-ope rat 1ve 

- Race bias - responding differently to a person of one's race 
rather than another race 

- Straight jacket Interview, I.e. questions that elicit yes/no 
responses rather than encourage a flow of Ideas/explanation 

- Respondent bias, 1n which the respondent would give a response 
that reflects favourably on h1m/herself rather than the true 
situation. 

(Wragg, 1984:176-177) 



Moser & Kalton (1979: 276-7) brought up the problem of Inadequate response 1n 
interviews. Quoting Kahn and Cannell (1957) which gives five principal symptoms 
of Inadequate response. 

(The) partial response 1n which the respondent gives a relevant but 
incomplete answer; non-response, when the respondent remains silent or 
refuses to answer the question; irrelevant response, in which the 
respondent does not answer the question asked; inaccurate response, when 
the question is answered by a reply which is biased or distorted; and the 
verbalized response problem, when the respondent explains why he cannot 
answer the question, perhaps because he fails to understand it, because he 
lacks the information necessary for answering it, or because he thinks it 
is irrelevant or inappropriate. 



In cases mentioned above, interviewer skills should come into play. The 
interviewer would also have to tread a fine line between pressing for the 
response and information that is needed, or knowing to leave well enough alone, 
in order to maintain goodwill, and perhaps better cooperation at another time. 



Analysis of interviews 

Among the hints Burgess (1984:121) gives for analysis when an audio- recording is 
used, is 

First, the interview needs to be played before transcription begins. 
Secondly, an index should be prepared using the counter on the tape 
recorder so that a record is kept of the themes that occur throughout the 
tape. Thirdly, only relevant materials should be completely transcribed. 



Administration of interviews 

Janet Hooper (personal conversation) carried out three interviews with teacher 
respondents 1n the Southampton University study of explicit knowledge about 
language (Mitchell, Hooper & Brumflt, 1994). These were: 

- One before starting the observation 

- One Immediately after the observation 

- One a term later. 
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A sequence of Interviews allows a further range of perspectives to be developed 
on the same phenomenon. 

In the proposed study, the following will be carried out: 

- One before starting the observation 

- One halfway through the observation 

- One immediately after the observation. 

Thus three different viewpoints should be gained from the same subjects. 



Conclusion ' 

This paper has tried to take the reader through the process of justifying 
particular research techniques, using the Malaysian research as a specific 
illustration. In doing this, much of the research literature on ethnography, 
systematic observation, questionnaires, and Interviewing has been briefly 
surveyed. Further discussion of these issues from a language teaching 
perspective can be found in Brumfit and Mitchell (1989), and Allwright and Bailey 
(1991). I hope that the general discussion will be useful for researchers with 
other fields of interest. 
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